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Abstract: The use of computer-aided detection (CAD) systems in mammography has been the subject of intense research 
for many years. These systems have been developed with the aim of helping radiologists to detect signs of breast cancer. 
However, the effectiveness of CAD systems in practice has sparked recent debate. In this commentary, we argue that com- 
puter-aided detection will become an increasingly important tool for radiologists in the early detection of breast cancer, but 
there are some important issues that need to be given greater focus in designing CAD systems if they are to reach their full 
potential. 
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Breast cancer is one of the leading causes of death among women in the United States. Screening 
mammography is currently the most effective tool for early detection of breast cancer. However, 
interpretation of mammograms can be difficult due to the variable appearance of normal breast tissue. 
Also, overlapping tissue structures can mask lesions in dense breasts making detection of some lesions 
difficult. Computer-aided detection systems (CAD) (Castellino, 2005; Sampat, Markey et al. 2005; 
Nishikawa, 2007) have been developed to assist radiologists in interpreting mammograms. Breast 
cancer may manifest in various findings — microcalcifications, masses, and architectural distortion. 
The main driving force of CAD research has been to detect early signs of these manifestations. The 
first FDA-approved CAD system for mammography was the R2 ImageChecker Ml 000 CAD system 
(U. S. Food and Drug Administration 1998) and since then the FDAhas approved other CAD systems 
for mammography such as the iCAD Second Look CAD system (U. S. Food and Drug Administration 
2002) and a mammography CAD engine from Eastman Kodak Company (U. S. Food and Drug 
Administration 2004). 

The effectiveness of CAD systems in detecting early cancer has received a lot of attention in recent 
years (e.g. (Health Imaging News 2008)). Even though CAD systems have been the subject of intense 
research, a question that one needs to ask is how effective are these systems in daily clinical use. The 
question is key since, as discussed by Krupinski (Krupinski, 2004), up until about 10 years ago, the use 
of CAD systems and their evaluation was limited to laboratory settings and most performance evalua- 
tions were carried out independent of the radiologist. Most research groups reported the performance 
of their system acting independently, as opposed to the performance of radiologists assisted by the 
system in a realistic clinical environment. However, more recently, the focus has shifted to observer 
performance studies in clinical environments. In one such study conducted by Freer and Ulissey (Freer 
and Ulissey, 2001), the effect of CAD on the interpretation of screening mammograms was assessed. 
The CAD system used in this study was the ImageChecker Ml 000 (version 2.0, R2 Technology) system. 
Perhaps, the most important component of this study was that it was conducted in a community breast 
center over a 12-month period and a total of 12,860 screening mammograms were interpreted with the 
CAD system. Freer and Ulissey concluded that the use of CAD in the interpretation of screening 
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mammograms can help increase the detection 
of early-stage malignancies without undue effect 
on the recall rate or positive predictive value 
for biopsy. 

Research groups working on CAD have been 
enthusiastic about the promise shown by CAD 
systems in detecting early cancer. However, in a 
recent controversial study, a multi-center research 
team led by Dr. Joshua Fenton of the University 
of California, Davis, found that CAD did not 
improve diagnostic accuracy. These results were 
published in the New England Journal of Medicine 
(NEJM) (Fenton, Taplin et al. 2007). Fenton et al. 
reported a decrease in diagnostic specificity from 
90.2% to 87.2% after incorporating CAD, a 
decrease in the positive predictive value from 
4.1% to 3.2%, and an increase in the biopsy rate 
by 19.7%. Fenton et al. concluded that there were 
no significant increases in sensitivity and the can- 
cer detection rate, and that the use of CAD was 
associated with significantly lower overall accu- 
racy. These conclusions sparked a huge debate on 
the overall efficacy of CAD systems. The study 
by Fenton et al. reported results from 43 facilities 
in three states, but it was only in seven of these 
facilities that CAD was actually used. All the 
seven facilities used the ImageChecker CAD sys- 
tem (version unspecified, R2 technology). Fur- 
thermore, out of the 429,345 mammograms 
included in the study, only 7% of the mammo- 
grams were read using CAD. Critics attacked the 
methodology adopted in the study, particularly the 
low number of mammograms actually read by 
CAD; they argued that in order to prove statistical 
significance of the efficacy of CAD, a much larger 
volume of cases with cancer is essential. Further, 
in an editorial of the same issue of NEJM, Hall 
(Hall, 2007) pointed out a possible flaw in the 
study by Fenton et al. — the time taken for a radi- 
ologist to get accustomed to a CAD device was 
not evaluated in this study and the adjustment to 
computer-aided detection has been estimated to 
take weeks to years. 

A study published by Mathew Gromet in 2008 
(Gromet, 2008) compares CAD to double reading 
of screening mammograms, offering a strong 
rebuttal to the study published by Fenton et al. 
(Fenton, Taplin et al. 2007). The CAD system used 
in this study was ImageChecker (version 5.3, R2 
Technology) and a total of 23 1 ,22 1 mammograms 
were reviewed between January 200 1 and December 
2005. Out of these, 112,413 cases (48.6%) were 



double-read and 118,808 cases (51.4%) were 
single-read with CAD. This makes the number of 
mammograms reviewed by CAD four times greater 
than what was reported by Fenton et al. (Fenton, 
Taplin et al. 2007). The results of the Gromet study 
indicate that CAD enhances performance of a 
single reader, yielding increased sensitivity with 
only a small increase in the recall rate. In another 
study, Skanne et al. (Skaane, Kshirsagar et al. 
2007) compared the use of a CAD system (Image- 
Checker, version 8.0, R2 Technology) to indepen- 
dent double reading of paired screen-film and full 
field digital screening mammograms (FFDM). The 
use of CAD resulted in an increase in the cancer 
detection rate for FFDM and for screen-film mam- 
mography in breast cancer screening performed 
with independent double reading. Yang et al. 
evaluated the performance of the Image Checker 
M 1 000 (version 3 . 1 , R2 Technology) CAD system 
applied to full-field digital mammograms for detec- 
tion of breast cancers (Yang, Moon et al. 2007). 
They reported a sensitivity of 95% on the fatty 
breast group and 98% on the dense breast group 
and reported an average of 1.80 false positive 
marks per patient on normal mammograms. In 
conclusion, the studies of Gromet (Gromet, 2008), 
Skanne et al. (Skaane, Kshirsagar et al. 2007) and 
Yang et al. (Yang, Moon et al. 2007), as well as a 
long history of encouraging laboratory studies, 
demonstrate the high potential of CAD for detect- 
ing early cancer. 

Even if a technology is effective in an absolute 
sense, it may not be cost-effective. The cost effec- 
tiveness of CAD with mammography for breast 
cancer screening is a question of great significance. 
A recent cost effectiveness study of adding a CAD 
system to a screening mammography program 
(Lindfors, McGahan et al. 2006) reported a mar- 
ginal cost per year of life saved (MCYLS) of 
$ 1 9,058. On the other hand, the MCYLS of screen- 
ing mammography alone was $16,023, which 
implies that the MCYLS is 19% greater for CAD 
added to screening mammography versus screen- 
ing mammography alone. However, one should 
not ignore the potential benefits of CAD in early 
detection of cancers. In the United States, investi- 
gators involved in cost-effectiveness studies often 
suggest that $50,000-$ 100,000 per life year saved 
may represent the upper limit of appropriate expen- 
diture. This study goes to show that in spite of an 
increase in the marginal cost of screening with 
CAD, it is still well within this acceptable range. 
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Given the broad scope of evidence to date that 
supports the potential of CAD, we feel confident 
that computer-aided detection will become an 
increasingly important tool for radiologists in the 
early detection of breast cancer. However, have all 
bases been covered in their design? We believe that 
there are still basic research challenges to be 
addressed. 

At a fundamental level, most CAD systems rely 
on image segmentation followed by feature extrac- 
tion and classification to characterize the lesion. 
While such an approach might work for lesions 
such as microcalcifications, for which there is a 
large contrast between the lesion and the back- 
ground, it is less effective for lesions that are more 
diverse in appearance, such as spiculated lesions 
and architectural distortion. The efficacy of CAD 
in detecting different types of breast cancer lesions 
has been debated. Birdwell et al. (Birdwell, Ikeda 
et al. 2001) conducted a study in which the char- 
acteristics of cancers missed at screening were 
studied and the ability of CAD (R2 Technology, 
version 2.0) to detect these cancers was assessed. 
In all, there were 110 patients with 115 cancers. It 
was found that on prior mammograms with missed 
cancers, 35 (30%) of the 115 lesions were calcifi- 
cations, and 80 of the 1 1 5 (70%) were mass lesions, 
with 32 of the 80 masses (40%) being spiculated 
or irregular. The most frequently suggested reasons 
for missed calcifications and masses were dense 
breast tissue and distracting lesions. CAD correctly 
identified 30 (86%) of 35 missed calcifications and 
58 (73%) of 80 missed masses. While CAD marked 
most of the cases that were missed by the radiolo- 
gists and needed a recall, the results also suggest 
that CAD systems are better at detecting microcal- 
cifications than masses. Clinical studies to evaluate 
the performance of commercial CAD systems for 
mass detection have reported sensitivities ranging 
from 67% to 89% with the false positive rate per 
image (FPI) ranging from 0.40 to 0.74. Thus, recent 
studies have emphasized algorithm development 
for specific lesion types that have traditionally been 
more difficult to detect. For example, Sampat et al. 
proposed a model-based framework for the early 
detection of spiculated lesions on mammography 
(Sampat, 2008). What makes their approach novel 
is that they measured the physical properties of 
spiculated lesions on a number of mammograms 
and attempted to develop a statistical model from 
those measurements. The statistical model serves 
as the basis for determining the parameters of 



several image-processing algorithms, which are 
deployed in sequence to aid in the detection of 
spiculated lesions and architectural distortion. 

A challenge facing CAD systems is the high 
number of false positive markings. Most CAD 
systems report good sensitivity but at the expense 
of high false positive rates. More research is 
required to reduce the false positive rate while 
maintaining a high degree of sensitivity. The 
research should take into account the human (radi- 
ologist) perception of these false positive mark- 
ings. There is a good case to argue that CAD 
systems are already reliable second readers. More 
often than not, when the computer marks false 
positives, the radiologists can recognize them as 
false positives rather easily and quickly, and dis- 
miss them, doing no harm to patient diagnosis. In 
the future, if the overall accuracy of CAD systems 
could be improved to approximate that of radi- 
ologists, the systems could be used as the first 
reader to triage cases most in need of review by a 
radiologist. While this is a sharp departure from 
the current status quo in mammo graphic CAD, 
clinical decision aids of this type are not unheard 
of in other areas of medicine, a good example of 
which is the Becton, Dickinson and Company 
intelligent pap imaging solution, which directs 
attention to slides that most likely contain abnor- 
malities (Becton, Dickinson and Company, 2008). 
Slides containing no abnormalities are classified 
under 'no further review required' category and 
can be archived without any further review by 
cytotechnologists. 

The radiology community is also exploring 
alternative and adjuvant imaging technologies for 
breast cancer detection, diagnosis, staging, and 
treatment monitoring. For instance, ultrasound 
(Kopans, 1998) is routinely used to further evalu- 
ate suspicious abnormalities identified on mam- 
mography. Ultrasound is particularly useful for 
distinguishing between cysts and solid lesions and 
for examining younger women with dense breasts. 
Likewise, dynamic contrast enhanced breast mag- 
netic resonance imaging (DCE-MRI) (Schnall, 
2001), digital breast tomosynthesis (DBT) 
(Niklason, Christian et al. 1997), digital breast 
computed tomography (DBCT) (Boone, Nelson 
etal; 2001; Yang, Carkaci et al. 2007) and 
stereomammography (SM) (Getty, Pickett et al. 
2001) are rapidly evolving technologies. DBT, 
DBCT and SM are being pursued with interest as 
they provide 3 -dimensional views of the breast, 
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thereby making up for a key limitation of x-ray 
mammography: loss of information due to the 
projection of 3-dimensional structures onto a 
2-dimensional image plane. CAD systems must be 
developed to meet the requirements of these new 
imaging techniques. One of the challenges facing 
the development of CAD systems for these new 
imaging modalities is the lack of a sufficient 
number of clinical cases for testing from these 
modalities as they are still in an evolving stage. 
Ideally, there should be a sufficient number of cases 
to allow for an unbiased development, training, 
and testing of CAD schemes. With an enlarged 
database, techniques can be properly optimized 
and useful features can be determined. 

More focus has to be given to standardized 
performance evaluation of CAD systems and 
regular quality assessment of CAD systems. 
Research studies on CAD systems have to be cen- 
tered on the needs of the radiologist and perfor- 
mance evaluation must be carried out with the end 
user in mind. The use of technologies such as eye 
tracking and visual prompts in the visual search of 
cancers on mammograms should be given more 
focus. In a study published by Hatton et al. (Hatton, 
Wooding et al. 2004), it was reported that visual 
attention was drawn to the prompted regions iden- 
tified by the ImageChecker CAD system (version 
unspecified, R2 technology). Different prompts 
were compared during screening and it was pos- 
sible to establish the impact of the prompts on 
visual search patterns. Solid non-transparent 
prompts were found to interfere with visual search 
patterns and distract radiologists while subtle 
prompts were found to be useful for radiologists 
interpreting screening mammograms. 

Finally, CAD systems need to be integrated into 
radiology training programs to help radiologists 
get comfortable with the systems. While evaluating 
the performance of CAD systems, it is essential to 
take into account the reader's training and experi- 
ence with CAD. CAD aided breast cancer detection 
training should become an important area of 
research focus. In one such study, Luo et al. (Luo, 
Qian et al. 2005) showed that there was a statisti- 
cally significant difference in each observer's 
performance in CAD-aided mammography inter- 
pretation before and after radiologists were trained 
on CAD. Luo et al. concluded that CAD training 
would influence perception, recognition, and inter- 
pretation of early breast cancer and CAD perfor- 
mance studies. 



In conclusion, while there is still debate on the 
magnitude of the impact of CAD systems cur- 
rently in clinical use, we cannot afford to ignore 
the potential benefits of CAD as we are facing a 
crisis in mammography in which women's access 
to mammographic screening is being endangered 
by a shortage of breast imaging specialists. In 
order for CAD systems to reach their full poten- 
tial, more emphasis must be given to CAD 
observer studies, developing novel methods for 
reducing the number of false positive detections, 
and integrating CAD into medical education. It 
is also important to bear in mind that the CAD 
systems are intended to assist radiologists, but 
not replace them. The radiologist should be the 
final judge in determining the final assessment. 
Any effort that assists radiologists in making more 
accurate interpretations should be encouraged. 
Failing to do so would greatly reduce the potential 
benefit to women's health, which may be improved 
in a significant way by the development of effec- 
tive CAD systems. 
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